Quantizing feature vectors in decision-making applications

ABSTRACT

A system, method and computer program product for classification of an analog electrical signal using statistical models of training data. A technique is described to quantize the analog electrical signal in a manner which maximizes the compression of the signal while simultaneously minimizing the diminution in the ability to classify the compressed signal. These goals are achieved by utilizing a quantizer designed to minimize the loss in a power of the log-likelihood ratio. A further technique is described to enhance the quantization process by optimally allocating a number of bits for each dimension of the quantized feature vector subject to a maximum number of bits available across all dimensions.

GOVERNMENT LICENSE RIGHTS

This invention was made with U.S. Government support under Agreement No.HR0011-07-9-0002 awarded by DARPA. The U.S. Government has certainrights in the invention.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. §120 to U.S. patentapplication Ser. No. 11/383,199 filed May 13, 2006, the entire text ofwhich is specifically incorporated by reference herein.

FIELD OF THE INVENTION

The present invention relates generally to decision-making applications,and more specifically, verification systems that use statistical models.

BACKGROUND

Mobile devices, such as cellular telephones, are rapidly growing inprevalence. As such, users are beginning to expect not onlycommunication from mobile devices, but a wide variety of computationalservices as well. To facilitate this wider variety of services, it isoften beneficial to be able to determine the identity of a speaker basedon that person's voice. Making this determination is known in the art asspeaker verification. Speaker verification is possible because thepatterns of acoustical energy which constitute human speech vary fromperson to person.

Speaker verification offers many benefits to the user interface ofmobile devices. Speaker verification may allow authenticating users foraccess to computational services offered via mobile devices. It can alsobe used to personalize the user experience for the specific individualaccessing the service. It is noted that speaker verification also mayenhance systems other than mobile devices. Broadly speaking,speech-based interfaces are suitable for a wide variety of access andoperational requirements. Speaker verification enhances any speech-basedinterface, thus helping adapt systems with traditional user interfacesto work well under unusual operational requirements. For example, aspeech-based interface with speaker verification may facilitate a humanwho does not have full use of his hands in using a computer systemnormally operated via a keyboard and mouse. Furthermore, speakerverification may be used to accomplish biometric authentication on anytype of system regardless of its user interface.

SUMMARY OF THE INVENTION

One exemplary aspect of the present invention is a method forclassification of an analog electrical signal using statistical modelsof training data. In this method, an analog electrical signal isconverted into a sequence of multi-dimensional feature vectors. Thefeature vectors are quantized on a relevant subspace of a firststatistical model as a function of a second statistical model. Alikelihood ratio is determined by dividing the likelihood that thequantized feature vector belongs to the second statistical model by thelikelihood that the quantized feature vector belongs to the firststatistical model. The logarithm of the likelihood ratio is computed.The analog electrical signal is classified as belonging to the secondstatistical model if the average of the logarithms of likelihood ratiosexceeds a predefined classification threshold.

Another exemplary aspect of the present invention is a system forclassification of an analog electrical signal using statistical modelsof training data. In this system, an extraction unit is configured toconvert an analog electrical signal into a sequence of multi-dimensionalfeature vectors. A scoring unit is configured to determine a relevantsubspace from a first statistical model based on maximum likelihood thatthe subspace accurately represents the feature vector. A quantizing unitis configured to quantize the feature vector according to the relevantsubspace and a bit allocation strategy vector specifying the number ofbits to be allocated for each dimension. A calculating unit isconfigured to compute the likelihood ratio by dividing the likelihoodthat the quantized feature vector belongs to a second statistical modelby the likelihood that the quantized feature vector belongs to the firststatistical model. The calculating unit further computes the logarithmof the likelihood ratio. A determining unit is configured to average thelogarithms of the likelihood ratios of a predetermined number of framesand to further determine whether the resulting average exceeds apredefined classification threshold. A bit allocation unit is configuredto determine the bit allocation strategy vector by minimizing aLaGrangian cost function.

Yet a further exemplary aspect of the invention is a computer programproduct. The computer program product includes computer readable programcodes configured to cause the program to: convert an analog electricalsignal into a sequence of multi-dimensional feature vectors; quantize afeature vector on a relevant subspace of a first statistical model as afunction of a second statistical model; determine a likelihood ratio bydividing the likelihood that the quantized feature vector belongs to thesecond statistical model by the likelihood that the quantized featurevector belongs to the first statistical model; compute the logarithm ofthe likelihood ratio; average a series of log-likelihood ratios; andclassify the analog electrical signal as belonging to the secondstatistical model if the average of the logarithms of likelihood ratiosexceeds a predefined classification threshold.

The foregoing and other features, utilities and advantages of theinvention will be apparent from the following more particulardescription of various embodiments of the invention as illustrated inthe accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an exemplary system for distributed speaker verification,the system embodying the present invention.

FIG. 2 demonstrates an exemplary configuration of a front-end unit.

FIG. 3 demonstrates an exemplary configuration of a speaker verificationunit.

FIG. 4 shows an exemplary flowchart of the operations performed by thepresent invention to classify an analog electrical signal.

FIG. 5 shows experimental results of the present invention againstconventional approaches.

FIG. 6 shows an illustrative embodiment of a computer system inaccordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The following description details how the present invention is employedto classify an analog electrical signal using statistical models oftraining data. Other applications of the present invention utilizing thegeneral principles set forth herein are also contemplated. Thus, thepresent invention should not be considered limited to the configurationsshown and discussed herein. Throughout the description of the inventionreference is made to FIGS. 1-6. When referring to the figures, likestructures and elements shown throughout are indicated with likereference numerals.

As will be appreciated by one skilled in the art, the present inventionmay be embodied as a method, system, or computer program product.Accordingly, the present invention may take the form of an entirelyhardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,the present invention may take the form of a computer program product ona computer-usable storage medium having computer-usable program codeembodied in the medium.

Any suitable computer usable or computer readable medium may beutilized. The computer-usable or computer-readable medium may be, forexample but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or propagation medium. More specific examples (a non-exhaustive list) ofthe computer-readable medium would include the following: an electricalconnection having one or more wires, a portable computer diskette, ahard disk, a random access memory (RAM), a read-only memory (ROM), anerasable programmable read-only memory (EPROM or Flash memory), anoptical fiber, a portable compact disc read-only memory (CD-ROM), anoptical storage device, a transmission media such as those supportingthe Internet or an intranet, or a magnetic storage device. Note that thecomputer-usable or computer-readable medium could even be paper oranother suitable medium upon which the program is printed, as theprogram can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this document, a computer-usableor computer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, or device.The computer-usable medium may include a propagated data signal with thecomputer-usable program code embodied therewith, either in baseband oras part of a carrier wave. The computer usable program code may betransmitted using any appropriate medium, including but not limited tothe Internet, wireline, optical fiber cable, RF, etc.

Computer program code for carrying out operations of the presentinvention may be written in an object oriented programming language suchas Java, Smalltalk, C++ or the like. However, the computer program codefor carrying out operations of the present invention may also be writtenin conventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

FIG. 1 shows an exemplary identification system 102 embodying thepresent invention. The system 102 includes a speaker 104. A goal of thesystem may be to verify the identity of the speaker 104. Thisverification need not depend on a standard of absolute certainty.Rather, it may instead be based on statistical techniques that determinewhether the identity of the speaker is known with a desired probability.Authenticating the speaker's identity may be a prerequisite to allowingthe speaker access to other functionality offered by the system. It iscontemplated that the speaker 104 is a human being, as the presentinvention capitalizes on the differences in speech patterns betweenhuman beings to distinguish between the humans themselves. However, thisis not a prerequisite of the present invention.

The speaker 104 emits acoustical energy 106 (e.g., sound.) It isemphasized that while the exemplary system is based on acousticalenergy, the present invention can be used to classify various realevents using statistical modeling. For example, the present inventionmay be utilized to analyze optical energy 103 (e.g., light) using aoptical transducer 105 in order to perform face recognition, a biometrictechnique to verify a human's identity based on the uniquecharacteristics of his or her face. The present invention may also beutilized to analyze optical energy 103 to perform optical characterrecognition, a technique to determine the identity of characters in acharacter set based upon their shapes.

The energy 106 is received by a front-end unit 108. The front-end unitmay be any technological device suitable for receiving and processingenergy. For example, the front-end unit may be a smart phone. Thefront-end unit may also be a peripheral device capable of receiving theenergy, coupled to a computer system capable of processing the energy.

Broadly speaking, the front-end unit 108 is responsible for convertingthe energy 106 into a compressed format suitable for analysis todetermine the identity of the speaker 104. The front-end unit 108 mayadditionally perform other administrative functions. The front-end unit108 can incorporate a transducer (such as a microphone) to convert theenergy into an analog electrical signal. Once the acoustic energy isconverted into electrical energy, a series of frames are generated, witheach frame measuring the signal over a specific time interval. Asdiscussed in more detail below, the front-end unit 108 converts eachframe into a feature vector. Each feature vector is a multi-dimensionalset of values representing the characteristics of the frame. Thefront-end unit then quantizes the feature vectors. Quantization is atype of compression designed to beneficially maximize the degree ofcompression of the feature vector while minimizing the degree ofdiminution in the ability to verify the speaker.

The front-end unit transmits the quantized feature vectors 110 to thespeaker verification unit 114. This transmission may be effected by anetwork 112. The network may be any of various types known in the art,including by example and without limitation TCP/IP, GSM®, Wi-Fi®,Bluetooth® piconets, token ring and microwave. GSM is a registeredtrademark of the GSM Association, London, United Kingdom. Wi-Fi is aregistered trademark of the Wi-Fi Alliance, Austin, Tex., United States.Bluetooth is a registered trademark of Bluetooth SIG, Inc., Bellevue,Wash., United States. It is noted that the network 112 may be wirelessand may be part of the broader global telecommunications network.

The speaker verification unit 114 receives the quantized feature vectors110. As discussed in detail below, for each quantized feature vector110, a ratio is computed between the probability that the feature vector110 belongs to a speaker under consideration and the probability that itbelongs to the Universal Background Model (UBM). The UBM is arepresentation of a typical, average speaker, designed to have as fewidiosyncrasies as possible. The resulting ratio is known as thelikelihood ratio. The logarithm of this value is taken; the resultingvalue is known as the log-likelihood ratio. The resulting values areaveraged for a predetermined number of feature vectors. This average maybe compared to a threshold value. If the average exceeds the threshold,the speaker verification unit 114 classifies the quantized featurevectors as belonging to the speaker 104. The system 102 can beconfigured to simultaneously consider multiple potential speakers aspossible identities of the actual speaker 104. In this case, the averageof the log-likelihood ratio may be calculated and compared separatelyfor each speaker under consideration.

The speaker verification unit 114 transmits directives to the front-endunit 108 concerning more efficient and/or more appropriate ways toperform the quantization process. Such directives may be transmitted viathe network 110. In particular, the speaker verification unit 114 mayprovide a special directive known as the bit allocation strategy vector(BASV) 116. As discussed in detail below, the BASV 116 is a direct inputto the quantization process. It improves the quantization process byspecifying an optimal manner of quantizing the feature vectors.

It is noted that the speaker verification unit 114 may be responsiblefor verifying speakers for a large number (possibly thousands) offront-end units 108. Moreover, the network 110 may have limitedbandwidth, particularly if it is part of, or implemented via, thebroader telecommunications system. Quantization of the feature vectorsis beneficial because reducing the size of the feature vectorssignificantly reduces both the load on the speaker verification unit andthe required network bandwidth, thus improving these potentialbottlenecks in the system.

Turning now to FIG. 2, an exemplary configuration of a front-end unit108 is illustrated. The configuration shown is used to demonstrate theoperation of the present invention. Energy 106 is received at thefront-end unit 108 by a transducer 202. As previously noted, the presentinvention can process a wide variety of types of energy. In a furtherembodiment of the present invention, the energy is acoustical energy,and the transducer is an acoustical transducer. In another furtherembodiment of the present invention, the energy is optical energy, andthe transducer is an optical transducer. The transducer converts theenergy into an analog electrical signal 204.

The analog electrical signal 204 is received by the extraction unit 206.The extraction unit 206 may convert the signal into speech frames. Aframe is a sample of incoming speech data over a specific time interval.Frames may, for example, be sampled at intervals of 10 millisecondseach. It is noted that in some embodiments of the present invention,frames may overlap.

The extraction unit 206 may further extract specific data from eachinput speech frame in order to generate a multi-dimensional featurevector 208. Feature vectors may be implemented as vectors of scalarvalues. Each scalar value is referred to as a dimension. The dimensionsmay be floating-point numbers and may each be 32 bits in length. In anembodiment of the present invention, Mel-frequency cepstral coefficients(MFCC's) are extracted from the input speech frames. Each MFCC may berepresented via a single dimension. See G. N. Ramaswamy, J. Navratil, U.V. Chaudhari and R. D. Zilca, “The IBM System for the NIST 2002 CellularSpeaker Verification Evaluation,” in IEEE ICASSP, Hong Kong, April 2003,incorporated herein in its entirety by reference. The logarithmic energyof the input speech frame may also be extracted into a single dimension.The first derivatives and even the second derivatives of any of thepreceding measurements may be taken, with each such derivative itselfpossibly constituting another dimension. It is emphasized that themeasurements which may be used as dimensions of the feature vector arenot limited to those discussed here. As an example of types of datatypically extracted from input speech frames, the ETSI Aurora, astate-of-the-art digital speech recognition system, extracts 13 MFCC'sand the logarithmic energy from each input speech frame. See ETSI ES 201108 V1.2.2, “Speech Processing, Transmission and Quality Aspects (stq);Distributed Speech Recognition; Front-End Feature Extraction Algorithm;Compression Algorithms,” Tech Rep., ETSI, 2000, incorporated herein inits entirety by reference.

The multi-dimensional feature vector 208 may be transmitted to a scoringunit 210. The scoring unit 210 may use statistical techniques todetermine the nature of the feature vector. Specifically,diagonal-covariance Gaussian Mixture Models (GMM's) may be used togenerate mathematical functions which determine the probability that aparticular feature vector 208 represents a particular speaker. Suchfunctions are referred to as likelihood functions.

In a particular embodiment of the invention a first statistical model,which may be based on GMM's, is known as the Universal Background Model(UBM) is used. The UBM is intended to represent a typical, averagespeaker. It may be designed to have as few idiosyncrasies as possible,and may in fact be trained based upon an average of many differentreal-life speakers. It is noted that the UBM may be considered as asingle speaker for purposes of the likelihood functions, even if it isin reality a composite of numerous speakers. A second statistical model,which may also be based on GMM's, is used to model a given speaker. Thissecond model may be generated by Bayesian adaptation of the UBM.

In an embodiment of the present invention, the mean vectors of thesecond statistical model are derived from the first speaker model viaMAP adaptation based on speaker-specific training data. It iscontemplated that the weights of each speaker and the diagonalcovariance matrices are identical between the two statistical models.See D. A. Reynolds and T. F. Quatieri and R. B. Dunn, “SpeakerVerification using Adapted Gaussian Mixture Models,” Digital SignalProcessing, vol. 10, no. 1-3, 2000, incorporated herein in its entiretyby reference.

The scoring unit 210 may incorporate a copy of the first statisticalmodel 212, the UBM. The scoring unit 210 determines which subspace ofthe UBM has the highest probability of having generated themulti-dimensional feature vector 208. This subspace may be a singleGaussian element of a Gaussian Mixture Model. It is contemplated thatthe selected subspace will have the same number of dimensions as thefeature vectors. It is noted that each Gaussian element of a GMM canhave an index which uniquely identifies it. As will be discussed in moredetail below, the speaker verification unit (item 114 of FIG. 1) caninclude an independent but identical copy of the UBM. In this case, theidentity of the selected subspace can be accurately communicated usingthe index alone.

The multi-dimensional feature vector 208 and the index of the selectedsubspace may be transmitted to a quantizing unit 214. The quantizingunit 214 may quantize the feature vector 208 based on the selectedsubspace. Quantization is a compression process which may reduce thenumber of bits with which a given dimension of the feature vector isexpressed. Each possible value of the original dimension may map to adeterminate quantized value. Conversely, each possible value which canbe expressed using the number of bits allocated to the reduced dimensionmay represent a range of possible values of the original dimension. Itis noted that a higher number of bits allows for greater accuracybecause more possible values can be represented. The number of valueswhich can be represented is 2^(b), where b is the number of bits inwhich the dimension is represented.

In an embodiment of the present invention, a dimension may berepresented using zero bits, which is to say it is omitted entirely fromthe quantized feature vector. This may be beneficial if the dimension isof minimal importance in verifying a particular speaker. Conversely, inanother embodiment of the present invention, a dimension of thequantized feature vector may contain the same number of bits as theoriginal feature vector (the dimension is not compressed). However, itis contemplated that in practice, this extreme case is unlikely. Indeed,it is contemplated that quantization will significantly reduce thenumber of bits of any given dimension of the feature vector.

The goal of quantization is not merely to perform a simple roundingoperation, but rather to encode the data in a manner which minimizes theaverage distortion of the data. The quantizing unit 214 may thusiteratively compute encoder partitions based on a nearest neighborcondition. The resulting partitions are intended to be reconstructed atthe speaker verification unit according to the centroid condition.

The total number of bits in which the quantized feature vector isrepresented may be predefined. A bit allocation strategy vector (BASV)116 may be used to reallocate the number of bits used by each dimensionof the feature vector, subject to this maximum. The quantizing unit 214may receive an initial BASV and may quantize subsequently receivedfeature vectors 208 so that each dimension is represented using thenumber of bits specified by the BASV 116. It is noted that the optimalBASV depends on the set of speakers under consideration. Thus, in anembodiment of the present invention, the quantizing unit may receive anupdated BASV 116 at any time. Feature vectors subsequently received maybe quantized according to the updated BASV 116.

The quantizing unit 214 performs the quantizing operation in a mannerintended to minimize a power of the loss in the log-likelihood ratiobetween the original feature vector and the quantized feature vector. Inan embodiment of the present invention, this power is two.Alternatively, the quantizing operation may be performed using aweighted Mean Squared Error (MSE) quantizer. As is demonstrated below, aweighted MSE quantizer minimizes the loss in the log-likelihood ratio.This is the case whether or not multiple speakers are underconsideration.

The quantized feature vectors 110 generated by the quantizing unit 214are transmitted to the speaker verification unit 114. The index 216 ofthe selected subspace determined by the scoring unit 210 may betransmitted along with the quantized feature vectors.

Turning now to FIG. 3, an exemplary configuration of a speakerverification unit 114 is illustrated. Quantized, multi-dimensionalfeature vectors 110 and the index of the selected subspace of the firststatistical model 216 are received at a calculating unit 302. Broadlyspeaking, the calculating unit 302 is responsible for performingcalculations related to the likelihood that the quantized featurevectors represent one of a set of speakers under consideration. Thecalculating unit 302 may include a first statistical model 304. Asdiscussed above, the first statistical model 304 may be the UniversalBackground Model. It is contemplated that this copy of the UBM isidentical to that found in the front-end unit (item 108 in FIGS. 1 and2). The calculating unit 302 may also incorporate second statisticalmodels 306. Each such second statistical model represents a speakerunder consideration.

The calculating unit 302 utilizes the likelihood function for the secondstatistical model 306, along with the selected subspace, to determinethe probability that a quantized feature vector represents the speaker.It further utilizes the likelihood function for the first statisticalmodel 304, along with the selected subspace, to determine theprobability that the same quantized feature vector represents thebackground (e.g., a hypothetical average speaker.) The calculating unit302 divides the figure for the second statistical model by the figurefor the first statistical model, yielding a result known as thelikelihood ratio. This ratio is relevant because even if a quantizedfeature vector represents a given speaker with a probability of, forexample, 0.6, the speaker verification unit 114 may nonetheless concludethat it does not represent that speaker if it also calculated aprobability of 0.8 of representing the background. The calculating unitmay further take the logarithm of the likelihood ratio. For brevity,this result may be referred to as the log-likelihood ratio. Furthermore,if more than one potential speaker is under consideration, this processmay be repeated for each target speaker.

The calculating unit 302 may consult a lookup unit 308 in lieu ofperforming the calculations in real time. This beneficially reduces theamount of time required to perform speaker verification. In anembodiment of the present invention, the lookup unit 308 containsprecomputed values of the likelihood functions. Determining thelog-likelihood ratio thus requires two table lookups, a divisionoperation and a logarithm operation. In another embodiment of thepresent invention, the lookup unit 308 contains precomputed values ofthe log-likelihood ratio itself. Determining the log-likelihood ratiothus requires only one table lookup with no further calculation. Animplementation of the present invention may thus select whicheverembodiment is more appropriate to that implementation's operationalconstraints.

The calculating unit 302 may transmit the log-likelihood ratios to adetermining unit 310. The determining unit 310 receives the ratios and,for each speaker under consideration, averages together the values asthey are received. In an embodiment of the present invention, theaveraging operation used is the arithmetic mean. In another embodimentof the present invention, the averaging operation used is the geometricmean. The determining unit 310 may continue this process until thevalues for a predefined number of quantized feature vectors for eachspeaker under consideration are averaged.

When an average for a given speaker is finally determined, thedetermining unit 310 may compare it to a predefined classificationthreshold. Based on this comparison, the determining unit 310 may make adecision 312 as to whether the quantized feature vectors represent aparticular speaker under consideration. The decision may be representedby a Boolean value. The decision may be transmitted to the front-endunit 108 or to any other entity with a need to verify the speaker 104.

A bit allocation unit 314 may compute the bit allocation strategy vector(BASV) 116. The optimal BASV and the calculation of the log-likelihoodratios depend on the specific set of speakers under consideration.Therefore, the bit allocation unit may compute the optimal BASV for theinitial set of speakers. This computation is performed using LaGrangianoptimization. The bit allocation unit may also cause the values in thelookup unit 308 to be precomputed for the initial set of speakers. Itmay itself perform the appropriate calculations. It may also accomplishthis goal by providing directives to the calculating unit 302. Theoptimal BASV is transmitted to the front-end unit 108 so thequantization operation can be performed in the most optimal manner.Furthermore, whenever the set of speakers under consideration changes,such as when a new speaker is added to consideration or when a speakeris removed from consideration, the preceding steps may be repeated.

FIG. 4 shows an exemplary flowchart of the operations performed by thepresent invention to classify an analog electrical signal. It will beunderstood that each block of the flowchart illustrations, andcombinations of blocks in the flowchart, can be implemented by computerprogram instructions. These computer program instructions may beprovided to a processor of a general purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

At converting operation 402, an incoming analog electrical signal isconverted into a sequence of feature vectors. Each such feature vectoris multi-dimensional. The original multi-dimensional feature vectors canbe expressed as a sequence X={x_(i) ^(j)}. The elements of this sequenceare the vectors x_(i), where i ranges from 1 to N, N being the number offeature vectors in the sequence. It is noted that because each vectorx_(i) is derived from a different input speech frame, the vectors may beconsidered for purposes of mathematical calculations to be mutuallyindependent. The dimensions of the original feature vector having anindex of i are the scalars x_(i) ^(j), where j ranges from 1 to D, Dbeing the number of dimensions in each feature vectors. See G. N.Ramaswamy, J. Navratil, U. V. Chaudhari and R. D. Zilca, “The IBM Systemfor the NIST 2002 Cellular Speaker Verification Evaluation,” in IEEEICASSP, Hong Kong, April 2003, incorporated herein in its entirety byreference. Upon completion of converting operation 402, control passesto scoring operation 404.

At scoring operation 404, a relevant subspace from a first statisticalmodel is selected based on maximum likelihood that the subspaceaccurately represents the feature vector. The first statistical model isgenerated from a first training data set. It may be a Gaussian mixturemodel. It is represented in mathematical equations by λ_(B). Therelevant subspace may be a single multi-dimensional Gaussian element inthe first statistical model that is the most likely to have generatedthe multi-dimensional feature vector. The index of the relevant subspaceis represented by g_(i,j). After scoring operation 404 is completed,control passes to quantizing operation 406.

At quantizing operation 406, the feature vector generated by theconverting operation 402 is quantized on the relevant subspace of thefirst statistical model as a function of a second statistical model. Thesecond statistical model is generated from a second training data set.It may be a Gaussian mixture model. The second statistical model isrepresented in mathematical equations by λ_(S). The quantizing operationmay be performed according to a bit allocation strategy vectorspecifying the number of bits to be used for each dimension of thequantized feature vector.

Mathematically, the quantized feature vectors can be expressed as asequence {circumflex over (X)}={{circumflex over (x)}_(i) ^(j)}. Theelements of this sequence are the vectors {circumflex over (x)}_(i),where i ranges from 1 to N, N being the number of feature vectors in thesequence. The dimensions of the quantized feature vector having an indexof i are the scalars {circumflex over (x)}_(i) ^(j), where j ranges from1 to D, D being the number of dimensions in the feature vector.

The quantizing operation 406 is performed in a manner intended tominimize a power of the loss in the log-likelihood ratio between theoriginal feature vector and the quantized feature vector. In anembodiment of the present invention, the power is two. In this case, thequantizing unit attempts to minimize the following expression:(Λ(X)−Λ({circumflex over (X)}))²where Λ(X) represents the average of the log-likelihood ratios of theoriginal feature vectors and Λ({circumflex over (X)}) represents theaverage of the log-likelihood ratios of the quantized feature vectors.

Suppose that the MAP-adapted mean of the selected Gaussian with indexg_(i,j) in the second statistical model λ_(S) can be expressed as μ_(Sg)_(i,j) =μ_(Bg) _(i,j) +δ_(g) _(i,j) . Suppose also that the weightcoefficients w_(g) _(i,j) and covariance matrices Σ_(g) _(i,j) of thefirst statistical model λ_(B) and the second statistical model λ_(S) areidentical. Note also that the elements of the feature vector, of whichthere are D, are mutually independent. Under these assumptions, thelog-likelihood ratio Λ(x_(i) ^(j)) can be expressed using the followingequation:

$\begin{matrix}{{\Lambda\left( x_{i}^{j} \right)} = {\frac{\delta_{g_{i,j}}}{\sigma_{g_{i,j}}^{2}}\left\lbrack {x_{i}^{j} - \frac{\mu_{s_{{gi},j}} + \mu_{B_{{gi},j}}}{2}} \right\rbrack}} & (1)\end{matrix}$

The decision may be made after N feature vectors are processed. In thiscase, the average log-likelihood ratio Λ(X) over the N feature vectorscan be expressed with the following equation:

$\begin{matrix}{{\Lambda(X)} = {\sum\limits_{i = 1}^{N}{\sum\limits_{j = 1}^{D}{\frac{\delta_{g_{i,j}}}{\sigma_{g_{i,j}}^{2}}\left\lbrack {x_{i}^{j} - \frac{\mu_{S_{g_{i,j}}} + \mu_{B_{g_{i,j}}}}{2}} \right\rbrack}}}} & (2)\end{matrix}$

The elements of a quantized feature vector {circumflex over (x)}_(i)^(j) be statistically represented by a discrete probability massfunction. It can be demonstrated that the probability density function(pdf) can be used to reverse the quantizing process by computingΛ({circumflex over (x)}_(i) ^(j)). This is demonstrated by the followingequation:

$\begin{matrix}{{\Lambda\left( {\hat{x}}_{i}^{j} \right)} = {{\log\frac{P\left( {{\hat{x}}_{i}^{j}❘\lambda_{S}} \right)}{P\left( {{\hat{x}}_{i}^{j}❘\lambda_{B}} \right)}} \approx {\log\frac{P\left( {{\hat{x}}_{i}^{j}❘\lambda_{S}} \right)}{P\left( {{\hat{x}}_{i}^{j}❘\lambda_{B}} \right)}}}} & (3)\end{matrix}$

Thus, the loss in log-likelihood ratio between an unquantized featurevector x_(i) and its quantized form {circumflex over (x)}_(i) can beexpressed using the following formula:

$\begin{matrix}{L_{i} = {{\sum\limits_{j = 1}^{D}\left( {{\Lambda\left( x_{i}^{j} \right)} - {\Lambda\left( {\hat{x}}_{i}^{j} \right)}} \right)} = {\sum\limits_{j = 1}^{D}{\frac{\delta_{g_{i,j}}}{\sigma_{g_{i,j}}^{2}}\left( {x_{i}^{j} - {\hat{x}}_{i}^{j}} \right)}}}} & (4)\end{matrix}$

It is desirable to minimize the expected value of a power of the loss inthe log-likelihood ratio. If the power to be minimized is two, the valuewe wish to minimize can be expressed as follows:

${E\left( L_{i}^{2} \right)} = {{\sum\limits_{j = 1}^{D}{\sum\limits_{i = 1}^{N}{w_{g_{i,j}}\frac{\delta_{g_{i,j}}^{2}}{\sigma_{g_{i,j}}^{4}}{E\left\lbrack \left( {x_{i}^{j} - {\hat{x}}_{i}^{j}} \right)^{2} \right\rbrack}}}} + {\sum\limits_{j = 1}^{D}{\sum\limits_{k \neq j}{\sum\limits_{i = 1}^{N}{w_{g_{i,j}}\frac{\delta_{g_{i,j}}\delta_{g_{i,k}}}{\sigma_{g_{i,j}}^{2}\sigma_{g_{i,k}}^{2}}{E\left\lbrack {x_{i}^{j} - {\hat{x}}_{i}^{j}} \right\rbrack}{E\left\lbrack {x_{i}^{k} - {\hat{x}}_{i}^{k}} \right\rbrack}}}}}}$

Generally, the quantization operation does not change the mean. Underthis assumption, the second term (after the plus sign) is zero, and theexpression can be simplified to:

$\begin{matrix}{{E\left( L_{i}^{2} \right)} = {\sum\limits_{j = 1}^{D}{\sum\limits_{i = 1}^{N}{w_{g_{i,j}}\frac{\delta_{g_{i,j}}^{2}}{\sigma_{g_{i,j}}^{4}}{E\left\lbrack \left( {x_{i}^{j} - {\hat{x}}_{i}^{j}} \right)^{2} \right\rbrack}}}}} & (5)\end{matrix}$

It is noted that this expression represents a weighted Mean SquaredError (MSE) quantizer. Thus, the quantizing operation may be performedusing a weighted MSE quantizer.

It is contemplated that many systems embodying the present inventionwill be configured to classify the analog electric signal as one of (ornone of) any of a set of candidates. To allow for this possibility,separate log-likelihood ratios may be computed and averaged. Thedifference in log-likelihood ratio for the candidate m may be expressedas:L ^(m)=Λ^(m)(X)−Λ^(m)({circumflex over (X)})

Quantizing the feature vectors for each candidate under consideration isinefficient. Instead, it is desirable to determine a quantizer whichminimizes the expected value of a power of the loss in thelog-likelihood ratio across all the possible candidates. If the power ofthe log-likelihood ratio to be minimized is two:

${E\left\lbrack {\sum\limits_{m = 1}^{M}\left( L_{i}^{m} \right)^{2}} \right\rbrack} = {M{\sum\limits_{j = 1}^{D}{\sum\limits_{i = 1}^{N}{w_{g_{i,j}}\frac{\Delta_{g_{i,j}}}{\sigma_{g_{i,j}}^{4}}{E\left\lbrack \left( {x_{i}^{j} - {\hat{x}}_{i}^{j}} \right)^{2} \right\rbrack}}}}}$

where M is the total number of candidates under consideration and

$\Delta_{g_{i,j}} = {\frac{1}{M}{\sum\limits_{m = 1}^{M}{\delta_{m,g_{i,j}}^{2}.}}}$It is noted that this expression also represents a weighted Mean SquaredError (MSE) quantizer. Thus, the quantizing operation may be performedusing a weighted MSE quantizer even when multiple speakers are underconsideration.

A bit allocation strategy vector (BASV) is used to determine the numberof bits used for each dimension of the quantized feature vector. Thetotal number of bits used across all dimensions of the quantized featurevector may be fixed, while the number of bits per dimension may vary.The BASV is determined by minimizing a LaGrangian cost function subjectto the constraint of the received maximum number of bits per featurevector. A cost function is configured to represent the cost of reducingeach dimension to a certain number of bits. The cost is defined as theloss of log-likelihood ratio, taken to a predefined power, resultingfrom quantizing the feature vector so that each dimension has thespecified number of bits. In an embodiment of the present invention, thepower is two, e.g., the loss in log-likelihood ratio is squared. Thetotal cost is minimized subject to a constraint of a total number ofbits in which the quantized feature vectors should be expressed. In anembodiment of the present invention, the total number of bits ispredefined.

For a single candidate, the number of bits to be used by a given element{circumflex over (x)}_(i) ^(j) of the quantized feature vectors, giventhe selected subspace g_(i,j), may be determined by minimizing L_(i) ²subject to the constraint

$\left( {\sum\limits_{j = 1}^{D}b_{j}} \right) \leq {b_{q}.}$b_(j) represents the number of bits allocated to dimension j, whileb_(q) represents the total number of bits in which the quantized featurevector {circumflex over (x)}_(i) is expressed. L_(i) ² is the square ofthe loss of log-likelihood ratio as defined in (4) above. The LaGrangiancan thus be expressed as follows:

$C = {{\sum\limits_{j = 1}^{D}{\frac{\delta_{g_{i,j}}^{2}}{\sigma_{g_{i,j}}^{4}}{D_{j}\left( b_{j} \right)}}} + {\lambda\left( {\sum\limits_{j = 1}^{D}\left( {b_{j} - b_{q}} \right)} \right)}}$

Solving the LaGrangian by equating the first derivative with respect tothe bit allocation variables to zero and using equation (4), the optimalnumber of bits per dimension for the selected subspace g_(i,j) is asfollows:

$\begin{matrix}{b_{g_{i,j}}^{j} = {{\frac{1}{D}b_{q}} + {\frac{1}{2}\left\lbrack {\frac{\delta_{g_{i,j}}^{2}}{\sigma_{g_{i,j}}^{2}} - {\frac{1}{D}{\sum\limits_{k = 1}^{D}{\log_{2}\left( \frac{\delta_{g_{i,k}}^{2}}{\sigma_{g_{i,k}}^{2}} \right)}}}} \right\rbrack}}} & (6)\end{matrix}$

For multiple candidates, we instead minimize

${\sum\limits_{m = 1}^{M}\left( L_{i}^{m} \right)^{2}},$obtaining the following expression:

$\begin{matrix}{b_{g_{i,j}}^{j} = {{\frac{1}{D}b_{q}} + {\frac{1}{2}\left\lbrack {\frac{\Delta_{g_{i,j}}}{\sigma_{g_{i,j}}^{2}} - {\frac{1}{D}{\sum\limits_{k = 1}^{D}{\log_{2}\left( \frac{\Delta_{g_{i,k}}}{\sigma_{g_{i,k}}^{2}} \right)}}}} \right\rbrack}}} & (7)\end{matrix}$

where M is the total number of candidates under consideration and

$\Delta_{g_{i,j}} = {\frac{1}{M}{\sum\limits_{m = 1}^{M}{\delta_{m,g_{i,j}}^{2}.}}}$

The number of bits in a given dimension of a quantized feature vectormust be an integer, as it is not possible to have fractions of a bit.However, equations (6) and (7) do not necessarily yield integers. Thefollowing algorithm may be used to convert the results of theseequations into a bit allocation strategy vector 116 specifying aninteger number of bits for each dimension. In the steps below, β={b_(g)_(i,1) ¹, . . . , b_(i,D) ^(D)} denotes the BASV.

1. Calculate └β┘ by rounding down each element of β. Specifically, eachelement is replaced with the greatest integer less than or equal toitself.

2. Determine a vector, (β−└β┘). It is noted that each element of thisvector will be greater than or equal to zero but less than one.

3. Select the top

$b_{q} - {\sum\limits_{j = 1}^{D}\left\lfloor b_{g_{i,j}}^{j} \right\rfloor}$elements from this vector and add one bit to each element thus selected.

At calculating operation 408, a likelihood ratio is determined bydividing the likelihood that the quantized feature vector belongs to thesecond statistical model by the likelihood that the quantized featurevector belongs to the first statistical model. The logarithm of thelikelihood ratio is also computed. After calculating operation 408 iscompleted, flow passes to averaging operation 410.

At averaging operation 410, the logarithms of the likelihood ratios asdetermined by calculating operation 408 are averaged over apredetermined number of frames. It is noted that because each frame isrepresented by a quantized feature vector, it suffices to simply countthe desired number of feature vectors and average over this number.Mathematically, the averaging operation may be expressed asΛ({circumflex over (X)}).

At determining operation 412, it is determined whether the averagedetermined by averaging operation 410 exceeds a predefinedclassification threshold. The threshold value may be a statisticallydetermined confidence level required to accept a hypothesis that thequantized feature vectors represent a specific candidate analog electricsignal. The threshold value may be expressed as Θ. Accordingly, in theevent that the average equals or exceeds the threshold:Λ({circumflex over (X)})≧ΛThe hypothesis is accepted. Thus, a decision is made that the featurevectors represent a candidate analog electrical signal. Conversely, inthe event that the average is less than the threshold:Λ({circumflex over (X)})<ΘThe hypothesis is rejected. Thus, a decision is made that the featurevectors do not represent the candidate signal.

FIG. 5 illustrates experimental results using the present invention onthe appropriately modified state-of-the-art IBM Speaker Verificationsystem.

The data consisted of the audio portion of the HUB4 Broadcast NewsDatabase (mono 16 kHz PCM). A subset of 64 speakers was selected as thetarget speaker set. A feature vector consists of 19-dimensional MFCC andtheir first derivatives1 with feature warping (i.e., D=38). A rate of100 frames per second, with 50% overlap was used and the MFCC werecomputed over a 20 millisecond window. For each speaker, two minutes ofdata were set aside and used for training the final models. The UBM,trained on independent broadcast news data, contained 256 38-dimensionalGaussian components. The speaker models, being MAP-adapted from the UBM,also had 256 components. For each speaker, 30 seconds (i.e., N=3000feature vectors) were used for testing performance.

Verification performance of the original system (unquantized features;32 bits per feature element) was compared against two quantizationmethods: (i) The quantizer of the present invention with variable bitallocation, and (ii) Conventional MSE quantizer designed from the UBMmodel with uniform bit allocation. Note that the latter approach doesnot require to pick the top-scoring Gaussian before quantization. Thus,the extra b_(c)=8 bits is assigned to the best feature elements. Thatis, those feature elements with the highest

$\frac{\delta_{g_{i,j}}}{\sigma_{g_{i,j}}^{2}}$ratio.

Experiments were run for various rate constraints. FIG. 5 shows thesingle-speaker verification performance for a 1:32 compression ratio(i.e., 1 bit per dimension, on average). The quantization method of thepresent invention performs extremely well considering that 1 bit perfeature element translates into being left/right of the correspondingGaussian's mean. One may suspect that most dimensions are useless suchthat most feature elements are simply skipped, allowing non-skippedfeature elements to use a relatively high number of bits.

Actually, most feature elements are indeed coded and only a few featureelements use a number of bits greater than one (maximum of 3 bits).Finally, note that the performance of the conventional MSE quantizer ispoor (increases the Equal Error Rate, or EER, by more than 18%). Similarconclusions were drawn for loser rate constraints and multi-speakerverification performance.

With reference to FIG. 6, an illustrative embodiment of a computersystem 602 in accordance with the present invention is shown. Onecomputer system 602 in which the present invention is potentially usefulencompasses a general-purpose computer. Examples of such computersinclude SPARC(r) systems offered by Sun Microsystems, Inc. andPentium(r) based computers available from International BusinessMachines Corp. and various other computer manufacturers. SPARC is aregistered trademark of Sun Microsystems, Inc. and Pentium is aregistered trademark of Intel Corporation.

The computer system 602 includes a processing unit 604, a system memory606, and a system bus 608 that couples the system memory 606 to theprocessing unit 604. The system memory 606 includes read only memory(ROM) 608 and random access memory (RAM) 610. A basic input/outputsystem (BIOS) 612, containing the basic routines that help to transferinformation between elements within the computer 602, such as duringstart-up, is stored in ROM 608.

The computer system 602 further includes a hard disk drive 614, amagnetic disk drive 616 (to read from and write to a removable magneticdisk 618), and an optical disk drive 620 (for reading a CD-ROM disk 622or to read from and write to other optical media). The hard disk drive614, magnetic disk drive 616, and optical disk drive 620 are connectedto the system bus 608 by a hard disk interface 624, a magnetic diskinterface 626, and an optical disk interface 628, respectively. Thedrives and their associated computer-readable media provide nonvolatilestorage for the computer system 602. Although computer-readable mediarefers to a hard disk, removable magnetic media and removable opticalmedia, it should be appreciated by those skilled in the art that othertypes of media that are readable by a computer, such as flash memorycards, may also be used in the illustrative computer system 602.

A number of program modules may be stored in the drives and RAM 610,including front end unit 108, a verification unit 114, and training data634. As discussed above, the computer system 602 is configured toclassify an analog electrical signal using statistical models oftraining data.

A user may enter commands and information into the computer system 602through a keyboard 636 and pointing device, such as a mouse 638. Otherinput devices (not shown) may include a microphone, modem, joystick,game pad, satellite dish, scanner, or the like. These and other inputdevices are often connected to the processing unit through a serial portinterface 640 that is coupled to the system bus 608.

A display device 642 is also connected to the system bus 608 via aninterface, such as a video adapter 644. In addition to the displaydevice, the computer system 602 may include other peripheral outputdevices (not shown), such as speakers and printers.

The computer system 602 operates in a networked environment usinglogical connections to one or more remote devices. The remote device maybe a server, a router, a peer device or other common network node. Whenused in a networking environment, the computer system 602 is typicallyconnected to the network 648 through a network interface 646. In anetwork environment, program modules depicted relative to the computersystem 602, or portions thereof, may be stored in one or more remotememory storage devices.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

Having thus described the invention of the present application in detailand by reference to embodiments thereof, it will be apparent thatmodifications and variations are possible without departing from thescope of the invention defined in the appended claims.

1. A method for classification of an analog electrical signal usingstatistical models of training data, the method comprising: convertingthe analog electrical signal into a sequence of feature vectors, thefeature vectors being multi-dimensional; quantizing a feature vector ona relevant subspace of a first statistical model as a function of asecond statistical model, the first statistical model generated from afirst training data set and the second statistical model generated froma second training data set; determining a likelihood ratio by dividingthe likelihood that the quantized feature vector belongs to the secondstatistical model by the likelihood that the quantized feature vectorbelongs to the first statistical model; determining a log-likelihoodratio by computing a logarithm of the likelihood ratio; averaging aseries of log-likelihood ratios; and classifying the analog electricalsignal as belonging to the second statistical model if the average ofthe logarithms of likelihood ratios exceeds a predefined classificationthreshold; wherein quantizing the feature vectors on the relevantsubspace includes minimizing a loss in a power of the logarithm of thelikelihood ratio between the original feature vector and the quantizedfeature vector.
 2. The method of claim 1, wherein the first statisticalmodel and the second statistical model are Gaussian mixture models. 3.The method of claim 2, wherein the subspace is a singlemulti-dimensional Gaussian element in the first statistical model thatis the most likely to have generated the multi-dimensional featurevector.
 4. The method of claim 3, wherein associating the relevantsubspace includes associating an index of the Gaussian utilized forquantization of the feature vector.
 5. The method of claim 1, whereinthe power is two.
 6. The method of claim 1, wherein quantizing thefeature vectors on a relevant subspace includes: receiving a maximumnumber of bits per feature vector; and allocating a number of bits foreach dimension of the feature vector by minimizing a LaGrangian costfunction subject to the constraint of the received maximum number ofbits per feature vector.
 7. The method of claim 6, wherein theminimization is based on the second statistical model.
 8. The method ofclaim 6, further comprising: rounding the number of bits for eachdimension of the feature vector such that each number of bits is thegreatest integer less than or equal to the number of bits determined byminimizing the LaGrangian cost function; counting the difference betweenthe total number of bits across all dimensions of the feature vectoryielded by the rounding operation and the received maximum number ofbits; and iteratively adding one bit to each dimension, in decreasingorder of the fractional number of bits removed by the roundingoperation, until the number of bits yielded by the determining operationhave been reallocated.
 9. The method of claim 1, further comprisingconverting acoustical energy into the analog electrical signal through atransducer.
 10. The method of claim 1, further comprising convertingoptical energy into the analog electrical signal through a opticaltransducer.
 11. A system for classification of an analog electricalsignal using statistical models of training data, the system comprising:a microphone configured to convert acoustical energy to the analogelectrical signal; an extraction unit configured to convert the analogelectrical signal into a sequence of feature vectors, the featurevectors being multi-dimensional; a scoring unit configured to determinea relevant subspace from a first statistical model based on maximumlikelihood that the subspace accurately represents the feature vector; aquantizing unit configured to quantize the feature vector according tothe relevant subspace and a bit allocation strategy vector; acalculating unit configured to compute the likelihood ratio by dividingthe likelihood that the quantized feature vector belongs to the secondstatistical model by the likelihood that the quantized feature vectorbelongs to the first statistical model and to further compute thelogarithm of the likelihood ratio; and a determining unit configured toaverage the logarithms of the likelihood ratios of a predeterminednumber of frames and to further determine whether the resulting averageexceeds a predefined classification threshold; wherein the quantizingunit is further configured to minimize a loss in a power of thelogarithm of the likelihood ratio between the original feature vectorand the quantized feature vector.
 12. The system of claim 11, furthercomprising a bit allocation unit configured to allocate a number of bitsfor each dimension of the feature vector by minimizing a LaGrangian costfunction subject to the constraint of a predefined maximum number ofbits per feature vector.
 13. The system of claim 12, wherein theminimization is based on the second statistical model.
 14. The system ofclaim 11, further comprising a lookup unit configured to store andretrieve pre-computed values of the logarithms of the likelihood ratios.15. The system of claim 11, wherein the power is two.
 16. A computerprogram product embodied on a non-transitory computer readable mediumcomprising: computer readable program codes coupled to thenon-transitory computer readable medium for classifying an analogelectrical signal using statistical models of training data, thecomputer readable program codes configured to cause the program to:convert the analog electrical signal into a sequence of feature vectors,the feature vectors being multi-dimensional; quantize a feature vectoron a relevant subspace of a first statistical model as a function of asecond statistical model, the first statistical model generated from afirst training data set and the second statistical model generated froma second training data set; determine a likelihood ratio by dividing thelikelihood that the quantized feature vector belongs to the secondstatistical model by the likelihood that the quantized feature vectorbelongs to the first statistical model and to further compute thelogarithm of this value; determine a log-likelihood ratio by computing alogarithm of the likelihood ratio; average a series of log-likelihoodratios thus determined; and classify the analog electrical signal asbelonging to the second statistical model if the average of thelogarithms of likelihood ratios exceeds a predefined classificationthreshold; wherein the computer readable program codes configured toquantize the feature vectors further comprise computer readable programcodes configured to minimize a loss in a power of the logarithm of thelikelihood ratio between the original feature vector and the quantizedfeature vector.
 17. The computer program product of claim 16, whereinthe first statistical model and the second statistical model areGaussian mixture models.
 18. The computer program product of claim 17,wherein the subspace is a single multi-dimensional Gaussian element inthe first statistical model that is the most likely to have generatedthe multi-dimensional feature vector.
 19. The computer program productof claim 18, wherein the computer readable program codes configured toassociate the relevant subspace further comprise computer readableprogram codes configured to associate an index of the Gaussian utilizedfor quantization of the feature vector.
 20. The computer program productof claim 16, wherein the power is two.